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An approach for developing examinations to assess the 
proficiency of health care professionals which utilizes a task 
inventory methodology is described. Examination development is hased 
upon the identification of health care functions which individuals 
within a specific category of health professional should be skilled 
in performing. The several hundred entries in the task inventory are 
organized under headings that correspond to components of the 
clinical problem- solving process. A subsequent stage in this 
methodology provides for the assignment of relative priority ratings 
to these health care functions, so that those receiving a high 
priority undergo further analysis in order to serve as the Jbasis for 
the development of examination materials. Priority ratings are 
determined by judgments regarding the expected frequency with which a 
health care function is performed and its "criticalness" to optimum 
health care delivery. Finally, the knowledge and skills components of 
each health care function receiving a high priority are specified and 
stated behaviorally , (Author/CK) 
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The devolopmont of examinations for health care professionals has Ioi'r been 
based upon curricular content: the subject matter that is presented and the 
way m wliich its presentation is structured within the curriculum. 

This paper describes an approach for developing examinations to assess the 
proficiency of health care professionals which utilizes instead a task inventory 
methodolosy. As a result, examination development is based upon the identification 
of health care functions whxch individuals within a specific category of health 
professional should be skilled in pcu-forming. The several hundred entries in the 
task inventory are organized under headings that correspond to components, of the 
clinical problem-solving process. 

A subsequent stage in this methodology provides for the assignment of relative 
priority ratings to these health care functions, so that those receiving a 
high priority undergo further analysis in order to serve as the basis for the 
development of examination materials. Priority ratings are determined by 
judgments regarding the expected frequency with which a health care function is 
performec and its criticalness to optimu-. health care delivery. 

Finally, the knowledge and skills components of each health care function receiv- 
ing a high priority are specified and stated behaviorally . These components 
become the performance criteria from which appropriate evaluation procedures 
are developed and minimum acceptable levels of proficiency are determined 
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OF HEALTH CAUL P110F];SS lONALS 

Harbnra J . Andi-cw , Ph . \) . 
National lionrd of Medical Lxamincrs^ 



Medical school faculties and external testing; agcncios 5;liare rcsponjaibil i ty for 
evaluating whether individuals wlio are undei-goinp trainin^c or who have completed 
training have acquired requisite proficiency in carrying out those functions whicli 
constitute their role in the health care delivery system. 

Whether the data from such evaluations are used to ar.sess individual competence 
or the effectiveness of the training programs themselves, the usefulness of 
these data and the appropr iatenej;s of the decisions made on the basis of them, 
depend upon the extent to which the evaluations assess relevant aspects of 
proficiency and the accuracy and reliability with which this is done. 

The purpose of this paper is to describe a methodology for the development of 
examinations designed to assess the proficiency of health care personnel. Tlie 
work described herein was undertaken by the Nationa*' Board of Medical Examiner. s 
as a part of its program to develop a national certifying examination for 
primary care physician's assistants.^ 

Major evaluations of individual proficiency usually occur at one or more key 
decision points along the continuum of training and experience* Tlio data from 
these evaluations are used as a significant factor in <ieciding vv'hethor an 
individual can be allowed to enter a more advanced phase of training, or whether 
he has developed sufficient proficiency to allow him to assume a position of 
l^rcater responsibility for health care delivery. 

As was sr^ggested earlier, a proficiency examinationj if it is to provide useful 
data fo^ decision-making, should be designed to assess those aspects of 
proficiency that are related to the health care functions in which a specific 
category of health professional needs to be skilled. 

Compo^i ents of Test Development 

For this reason, the following factors were taken into consideration in 
developing the proficiency examinations for primary care physician's assistants: 

1) Wliat functions should this type of health care provider be skilled in 
carrying out? 

2) What knowledge and skills are required in order to carry out each 
specific function? 

3) Wliat evaluation methodologies can be employed to assess the knowledge 
and skill components of the various health care functions? 
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Issues to be Considered 

Moreover, the following issues emerged, and nrn-cied to be effectively resolved: 

1) Could consensus be renchori concerning; the lioallh c:ire functions a 
prinuiry care physici:ui's ass5sla]\t ;;ho\ild be sl:illc(i in '">orf or in in 

2) \\l\tit would be the effect of the identi f ieat ion ol those fun .'.;s on 
the development of the physicirxn\s assistant concept? 

3) *How would the limitations on time and resources ii vail able for evaluation 
effect the design of the proficiency cxaniinat ior. itself? 

Rationa le 

The difficulty of achieving consensus among experts in a field has often been 
viewed as a serious obstacle to the definition c-f the proficiency that a 
graduate of medical school or a residency training proftram should possess in 
order to effectively assume responsibility fo: aticnt care. As of tliis writing, 
there are some forty-one educational institute training physician's assistants 
In curricula that vary from one to three ycar;^ in length, ar.d which accept 
students with diverse backgrounds in education and experience* 

Since available task analysis studies of what functions physician's assistants 
actually perform are limited in scope and sampling techniques, and since the 
utilization of physician's assistants (P. A*) can be expected to reflect 
evolutionary changes for some time to coine, it was felt that a methodology involv- 
ing the judgments of e>-perts in the field would be the most logical first step 
in identifying health care functions the P. A. should be skilled in performing. 
Moreover, since the P.A.'s proficiency would be viewed in terms of health care 
functions that constituted a body of •■'core*' proficiencies, educational programs 
would continue to enjoy flexibi 1 ity in the content and design of their curricula. 

Task Inventory Study 

In order to arrive at a consensus definition of the proficiency a primary care 
physician's assistant should bo expected to possess, it was decided that the 
health care functions a P,A, needs to be skilled in performing should first be 
identified, 

A review of existing task inventories concerning physician and non-physician 
health care providers was undertaken with the purpose of compiling a comprehen- 
sive list of health care functions.^ As a result of this rreview, over eight 
hundred health care functions were identified. 



•^T!ie following resources provided the most useful task inventory data: 

a) Golden, Archie S,, and Johnson, Robert. A Task Inventory for 
Non-Physician Health Teams in Primary Care > The Center Tor Allied 
Health Careers and the IlealtJi Services Research and Development 
Center, The Johns Hopkins Medical Institutions, February, 1972. 

b) Medex Program Patient Contact Record Keodback R eport > Health 
Services Manpower Brai^Ii, jNationnl Center for Health Services 
Research and Development, 

c) Physician Task Profile . Tcchnomics, Inc. , 1972* 

O 
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III order to orRnnlzo thowo functiotis for oa.sy revicnv by a [croup of exports, nnd 
to fncllitnlG the addition of any heallh care functians not on the li«t but 
relevant to the physician^s asfrjsl.anl, llicso several hiniclrud entries were 
organized into a larger taiik invent ory under tho lol lowing headini;s; 

I. DATA GATIII'JinNG 

• A. IflSTOllY AND PATIENT HHCOUDS 

B, rUVSICAL EX>\MINATI0N 

C, I.AnOKATORY TESTS AND INVESTIGATIVE PROCEDURES 

D, PATIENT MONITORING 

II. ANALYSIS AND INTERPRETATIOiN 
III. MEDICAL AND HEALTH CAUE STRATEGIES 

A. EMERGENCY PROCEDURES 

B. SURGICAL PROCEDURES 

C. CLINICAL 'PROCEDURES 

D. MANAGEMENT AND THERAPY 

A Group of twenty experts (members of the National }3oard Advisory Committee on 
the Physieian^s Assistant) Reviewed individually the task inventory of health 
care functions. These experts included physicians involved in the development 
of this new category of liealth care personnel, physician's assistants; physicians 
utilizing physician's assistants, physicians who train physicianVs assistants, 
and nurses concerned with the nurse practitioner concept, 

A sample group of tasks and the four rating categories which were developed are 
provided in Table 1. Each of the experts was asked to read and consider each 
of the functions listed on the thirty-throe page task inventory, and indicate 
by placing a check in the appropriate column to the right whether it was one the 
Type A primary care P.^. should definitely, probably, probably not , or definitely 
not be skilled in performing. (Since this examination was being developed to 
assess the proficiency of the Type A primary care physician's assistant, the 
conceptual definition of this type of P,A, developed by the National Academy of 
Sciences was used.) 

A fx^equency distribution of the twenty judgments made concerning each health 
care function was compiled for all functions listed on the task inventory. 
Each of the four rating categories was given a numerical value on a scale of one 
to four: a value of four being assigned to the ^*def initely category, and a 
Value of one to the '^definitely not " category. In turn, each judgment made by 
a rater was given a numerical value on the basis of the column into which a 
check had been. placed. 

The arithmetic mean of these judgments was then computed for each health care 
function. Those functions receiving a mean 'value of 3.50 - 4.00 were considered 
as ones that, in the view of expert opinion, a Type A primary care physicianVs 
assistant should definitely be skilled in performing. Functions receiving a 
mean value of 3.40 - 3,49 were reviewed by this expert group at a meeting during 
which the results of this task inventory study were presented. 

As a result of this task inventory study, over five hundred health care 
functions were identified as ones the primary care physicianVs assistant should 
definitely be. skilled in perf oi'ming. The relatively large number of functions 
was due to the fact that each one was stated quite specifically rather than as a 
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Example of Task Inventory Foj^mat with 
Snmplo Functioiis within Selected Catct,'orics 

PUOiJAnLV DKh'TNITELY 

i)KriNiTr;i,Y i^iionAUhY 'not not 



HISTOHY AND PATIENT KKCORDS: 
ask hifitory since la«t illness 


f — • — ■ — 
















write patient sximniarios 










PHYSICAL EXAMINATION: 










examine eyes using oplithalmoscope 


• 








test reflexes 










lAB TESTS/ INVESTIGATIVE PROCEDURES: 
perform white blood cell count 


















determine urine pH, sugar, protein 
iand specific gravj.ty 










take throat culture 










initiate request for chest x-ray 











Table 2 

Sample Task Inventory Entries and Format Used 
in the Priority Study 

FREQUENCY CR IT ICALJffiSS 





•high 






low 


high 






low 


laiERGENCY PROCEDURES: 

perfo^rm closed chest cardiac 
massage 


































control external hemorrhage 


















SURGICAL PROCEDURES: 


















suture minor lacerations 


















incise and drain abscess 


















excise wart surgically 


















CLINICAL PROCEDURES: 
apply and remove cast 


































obtain an electrccardioKram 


















COUNSELLING: 
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counsel for growth and 
development 


















-explain venereal disease 
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brord calonory. Tor example, the functions ro]alln(j to history tnkJnf; appear 
as ulevcn separate entries on the inventory, ran(;iiU^ from inciuirlny ahout the 
patient *s chief complaint to conduct in(^ a review ol systems. In a similar 
manner, wotind care including suturlnp; minor lacerations appears as four separate 
ent;rios: clean.se, irrii^atc, clobricJe, \und suture minor lacerations, 

It was felt that functions receiving a moan value of 2,50 - 3, '10 (and judged to 
be ones tlie pr iinary ' cai'c physician \q assistant should probably be skilled in 
performing) niiglit well rolled areas in which training profjrains differeci some- 
what in their eduoationa 1 objectives as well as (geographic variation.^ in health 
care delivery^. Since the CNamination was to be administered nationally, it 
seemed more appropriate for it to be dcsif^ncd to assess proficiencies that all 
pX'imary care physician's assistants should possess. 

In addition, the limitations of time and resources available for evaluation became 
the second major issue, and lead to the next phase of test development: a 
priority study. 

Priority Study 

A review of the health care functions identified from the task inventory study 
suggested that not all functions were equivalent in terms of their importance 
to the proficiency of the primary care physician *s assistant, Morecvor, the 
number of functions was so large that it was evident that no examination program 
could attempt to sample adequately the knowledge and skills related to all of 
them. For these reasons, a priority study was conducted using the same experts 
who had participated in the task inventory study, plus four additional experts, 
all of whom were pediatricians, ^ 

The purpose of the priority study was to establish the relative importance 
of the several hundred healtli care functions, so tliat. those receiving the highest 
rating in this regard wpuld serve as tlie primary focus for the development of 
examination for primary care physician's assistants. 

Two dimensions were selected as the basis for determining the priority of each 
health care function: 1) the frequency with which this function might be carried 
out in a primary care practicey and 2) its "criticalness" to optimum health care 
delivery. 

In order to determine the -priorities of the more than five hundred functions 
identified from the first phase of test development, a second task inventory 
was prepared . using the categories as listed on page thre^; The two dimensions 
which were to serve as the basis for arriving at these priority ratings (i,e,, 
frequency and criticalness) were represented by two scales, each consisting of 
four intervals. The anchor points of each scale were labeled *^high" and "low". 
Table 2 illustrates the format of the inventory used in this priority study. 

Each member of the group of expert judges was asked to consider the functions 
included on this second inventory and to indicate how frequently he felt the 
task would be performed in a primary care practice and how critical it was to 
optimum health care delivery, 

A frequency distribution of the judgments made regarding the frequency and 
criticalness of each health care function was compiled, A mean value for each 
health care function on each dimension was calculated by assigning a nul^^erical 
Value of 1 to 4 to the intervals on each scale as was done in the first task 
inventory study. 




The priority vnluo assiniicd to oach honltli care f\incLion was dotormi nod by usinf^ 
tho followiiiK roriimln: P = f -^ .2(0), whore 7 in ihc Jnoaji frcquoncy value and c 
is the mean criticalncss value, Tlic criticalnoss value v;as weichtcd more heavily 
than that of frequency because it wns rcconniVrOd Ihut wlijle some fitnctionr. arc 



pci'fornied infrequcMit ly (cpf., closed chest cardiac nu'iss.>ai;o) , 
lif e-and-dea th iiiiplica tionn when tliey arc reciuij'cd as a i)art 
delivery. 'fhe use of the above J'oi'Jnula resulted in a scale 
3,00 - 12.00. Those functions receiving? a priority value of 
further analy?.c<l, in the manner described below, . in order to 
for developing examination materials and evaluation i)rocedures. 



tliey often involve 
of liealtli care 
wliosc rauf^e was 
8.00 - 12.00 were 
serve as tho basis 



Specification of Performance Criteria 

Since the validity of an examination as an assessment of proficiency depends upon 
its capacity to evaluate accurately the knowledge and skills required to carry 
out specific health care functions, test committees were appointed to analyze 
these health care f uncti ens . and identify the knowledge and skills components 
related to each. These knowledge and skill components have been stated behavi orally 
to facilitate the selection of appropriate evaluation mechanisms and serve as 
the performance criteria to be used in assessing the proficiency of the P. A. 
Table 3 presents some of the criteria associated with tho following health care 
functions: wound care; suturing lacerations; taking an electrocardiogram; 
applying a plaster cast; physical examination. 

Table 3 

Sample Criteria for Selected 
Health Care Functions 



FUNCTION 



CRITERIA 



Wound Care 



Suturing Lacerations 



Electrocardiogram 



Cast Application 



Physical Examination 



a) recognize normal signs associated 
with wound healing 

b) identify lacerations requiring 
primary closure by suture 

a) select appropriate suture material 

b) effect a closure in which all dead 
space has been obliterated 

a) identify the correct placement of 
all V leads 

b) select appropriate courses of action 
to eliminate or minimize technical 
defects in an EKG tracing 

a) identify the correct position of 
functions for applying a forearm cast 

b) recognize errors made in, cast 
application 

a) recognize and name common skin lesions 

b) recognize and describe abnormalities 
seen on funduscoplc examination 



The methodology deucri.bed in this paper would appear to be applicable to any 
evaluation effort designed to assess the extent to which individuals have acquired 
necessary proficiency in carrying out specified health care functions. Although 
it was used here to assess the proficiency of physician's assistants, this 
methodolof^y could also be utilized at various points along the continuum of 
medical education for physicians and other health care professionals as a means 
of evaluating progress in attainiJig educational objectives. 



